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INTRODUCTION 


Breast  cancer  is  the  fourth  most  common  cause  of  death  among  women  in  the 
United  States  [1].  There  is  no  known  means  of  preventing  the  disease,  and  available 
therapy  has  been  of  very  limited  success  in  reducing  the  national  mortality  rate  over  the 
past  60  years.  Current  attempts  at  controlling  breast  cancer  concentrate  on  early  detection 
by  means  of  mass  screening,  using  clinical  breast  examination  and  periodic 
mammography,  because  ample  evidence  is  now  available  to  indicate  that  such  screening 
indeed  can  be  effective  in  lowering  the  death  rate  [2-13].  Cases  judged  abnormal  at 
screening  are  further  evaluated  with  breast  ultrasonography,  percutaneous-needle  and/or 
needle-localization  biopsy,  and  on  an  experimental  basis  with  breast  CT  and  breast  MRI. 

Extensive  mammography  quality  assurance  procedures  have  been  developed  over 
the  past  several  years,  and  now  are  mandated  by  the  U.S.  Mammography  Quality 
Standards  Act.  However,  these  procedures  for  the  most  part  cover  mammography 
equipment,  imaging  parameters,  and  image  processing.  They  have  not  yet  addressed  the 
issue  of  image  interpretation  in  a  meaningful  way. 

Breast  imaging  interpretation  is  taught  using  a  variety  of  approaches.  During 
radiology  residency  and  fellowship  one-on-one  instruction  is  used  widely,  but  this  is  not 
feasible  for  post-graduate  training  (required  for  all  radiologists  practicing 
mammography),  due  to  the  overwhelming  mismatch  in  numbers  of  teachers  and  students. 
As  a  result,  continuing  education  primarily  involves  group  instruction,  using  medical 
journals,  books,  lectures,  workshops,  and  videotapes.  A  particularly  effective  teaching 
approach  for  both  residency  and  post-graduate  training  involves  individual  review  of 
selected  case  material,  including  original  radiographs,  sonograms,  and  supporting  text 
material  that  indicates  eventual  clinical  diagnosis.  This  material  is  traditionally  presented 
in  the  format  of  film-based  teaching  files. 

However,  current  teaching  files,  because  they  involve  conventional  film  images, 
can  be  viewed  by  no  more  than  a  few  users  at  a  time.  Because  it  is  impossible  to  present 
the  imaging  work-up  of  each  case  on  an  image-by-image  basis  (one  cannot  anticipate  all 
the  work-up  options  that  might  be  selected  by  an  individual  user),  the  display  of  film 
cases  is  limited  to  sequential  descriptions  of  imaging  findings,  interpretations,  and 
pathologic  diagnoses. 

Digitally  stored  and  displayed  teaching  file  images,  on  the  other  hand,  can  be 
viewed  by  large  numbers  of  users  simultaneously,  at  different  workstations  (even  at 
distant  sites),  in  the  precise  sequence  in  which  the  images  were  obtained.  With  carefully 
structured  questions,  each  user  can  be  prompted  to  respond  by  making  his/her  own 
observations,  assessments,  and  work-up  decisions  just  as  if  the  patient  were  being 
examined  at  that  time.  This  effectively  replaces  a  “show  and  tell”  teaching  experience 
with  the  interactive,  response-driven  type  of  instruction  that  currently  must  be  taught  in 
person. 

Additional  advantages  of  a  digitally-based  teaching  file  are:  (1)  the  user  does  not 
have  to  sort  through  all  the  films  in  each  case  in  order  to  view  them  in  the  proper 
sequence;  (2)  the  user  is  not  burdened  with  the  request  to  replace  film  images  in  the  same 
sequence  in  which  they  were  presented;  (3)  images  and  supporting  text  material  cannot  be 
misfiled,  damaged,  lost,  or  stolen;  (4)  digital  images  are  much  more  amenable  to 
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post-acquisition  enhancement  (enlargement,  contrast  and  density  windowing,  filtering 
techniques  [edge  enhancement,  unsharp  masking,  etc])  than  are  film  images  (enlargement 
by  use  of  a  magnifying  lens);  (5)  digital  images  can  be  duplicated  easily  and  relatively 
inexpensively  without  loss  of  image  quality;  (6)  digital  images  can  be  viewed  in  various 
user-selected  sequences  (organized  by  mammographic  finding,  organized  by  pathologic 
diagnosis,  organized  by  degree  of  difficulty  of  interpretation,  organized  by  complexity  of 
work-up,  organized  by  use  of  specific  work-up  examinations);  (7)  user  progress  through 
the  teaching  file  can  be  linked  to  providing  correct  answers  for  key  questions;  and  (8) 
user  progress  through  the  teaching  file  can  be  documented  by  these  correct  answers, 
thereby  permitting  Category  I  credit  for  continuing  medical  education  to  be  awarded 
automatically  on  an  hour-for-hour  basis. 

This  funded  research  is  designed  to  investigate  the  effectiveness  of  an  interactive 
computer-based  digital  breast  imaging  teaching  file  (DBITF)  compared  to  that  of  an 
established  and  widely  used  conventional-film  teaching  file  (CFTF)  in  providing  initial 
and  continuing  medical  education  in  breast  imaging.  Two  hypotheses  will  be 
investigated:  (1)  A  totally  digital-based  breast  imaging  teaching  file  can  be  designed  and 
implemented  with  current  technology;  and  (2)  This  DBITF’s  interactive  response-driven 
type  of  instruction  is  a  more  effective  teaching  approach  than  the  traditional  passive 
“show  and  tell”  type  of  instruction  used  with  the  CFTF.  We  believe  that  the  DBITF  will 
succeed  primarily  because  it  has  the  advantage  of  more  closely  approximating  the 
real-life  work-up  of  patients,  since  the  user  is  presented  with  images  in  the  sequence 
originally  obtained  and  then  is  prompted  to  respond  to  his/her  own  observations  and 
arrive  at  work-up  decisions  as  if  the  patient  were  being  examined  at  that  time. 

BODY 

Six  tasks  were  outlined  in  the  Statement  of  Work  within  our  grant  proposal.  Almost  all 
of  the  Year  1  tasks  were  accomplished  as  initially  planned.  However,  two  minor 
modifications  in  our  original  plan  were  made  during  Year  1.  (1)  We  undertook  an  initial 
pilot  study  evaluation  of  the  DBITF  in  Year  1  rather  than  waiting  until  Year  3,  to  provide 
an  early  indication  of  effectiveness  and  user  acceptance  of  the  DBITF.  This  also 
permitted  us  to  incorporate  user  suggestions  for  improved  DBITF  performance  earlier  in 
the  process  of  DBITF  creation.  (2)  We  chose  not  to  restrict  case  selection  to  those 
already  entered  into  the  CFTF  as  of  the  end  of  Year  1 .  This  permitted  us  to  include  in  the 
DBITF  even  more  current  case  material  and  discussion  of  then  unrealized  concepts  and 
advances  in  breast  imaging.  As  a  result,  during  Year  1  we  selected  only  a  representative 
sample  of  CFTF  cases  for  inclusion  into  the  DBITF,  rather  than  all  1,000  cases  planned 
for  ultimate  inclusion.  The  minor  modifications  we  made  in  our  original  plan  were 
affirmed,  insofar  as  our  Year  1  annual  report  was  judged  “acceptable  as  written”. 

The  tasks  for  Year  2  outlined  in  the  Statement  of  Work  within  our  grant  proposal  were 
accomplished  exactly  as  initially  submitted.  In  our  Year  2  annual  report,  we  suggested  a 
single  modification  to  the  evaluation  (effectiveness  and  user  acceptance  analysis)  task 
planned  for  Year  3.  Specifically,  we  judged  it  impractical  to  divide  our  1,000  DBITF 
cases  into  two  500-case  sets  for  testing  on  radiology  residents  and  practicing  radiologists, 
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as  initially  planned,  simply  because  such  massive  case  sets  would  have  taken  many  days 
(probably  more  than  a  week)  for  each  individual  to  complete.  Realistically,  we  would 
have  been  able  to  recruit  few  if  any  volunteers  to  spend  this  much  time  on  the  formal 
evaluation  of  DBITF  versus  CFTF.  We  therefore  planned  to  select  representative  rather 
than  comprehensive  case  material  for  testing,  so  as  to  limit  the  evaluation  made  by  each 
individual  to  no  more  than  1  day.  This  minor  modification  was  affirmed,  insofar  as  our 
Year  2  annual  report  was  judged  “acceptable  as  written”. 

The  tasks  for  Year  3  outlined  in  the  Statement  of  Work  within  our  grant  proposal  were 
accomplished  exactly  as  initially  submitted,  except  for  the  minor  modification  (described 
above)  in  our  Year  2  annual  report. 

Task  1.  Select  cases  from  the  current  CFTF  for  inclusion  into  the  DBITF.  By  the 

end  of  Year  2,  we  completed  the  selection  of  all  1,000  planned  cases,  based  on  inclusion 
of  the  full  spectrum  of  mammographic  findings,  work-up  approaches,  and  disease  entities 
now  encountered  in  breast  imaging  practice.  Case  selection  includes  the  choice  of 
images,  collection  of  medical-related  data,  and  formulation  of  questions/answers  for 
interactive  response-based  instruction.  Since  many  selected  cases  represent  variations  in 
appearance  of  similar  mammographic  findings,  and  since  the  work-up  approaches  for 
some  mammographic  findings  are  limited  to  relatively  few  choices,  many  of  the  DBITF 
test  questions  have  similar  (at  times  identical)  wording. 

Task  2.  Convert  the  CFTF  to  digital  format.  In  Year  2  we  completed  the  digitization 
of  selected  film  mammograms  and  other  breast  images  (sonograms,  needle  localization 
images,  ductograms,  specimen  radiographs,  etc),  so  as  to  be  compatible  with  our  DBITF 
format.  Related  medical  data  (text)  also  was  incorporated  into  the  DBITF  format  that  we 
described  in  our  initial  grant  proposal.  As  we  continued  to  accrue  new  (current)  cases  in 
our  CFTF  during  Year  3,  we  incorporated  the  best  of  these  into  the  DBITF,  thereby 
replacing  older  cases  already  entered  into  the  DBITF.  We  did  not  include  cases  involving 
breast  CT  and  MRI,  since  the  clinical  roles  of  these  modalities  are  still  in  flux. 

Task  3.  Image  display.  We  have  completed  the  development  of  a  two-monitor  2K 
digital  image  display  workstation  (hardware  and  software),  and  have  tested  its  use  on  a 
variety  of  subjects  [14].  Refinements  in  software  were  based  primarily  on  the 
suggestions  of  our  initial  test  subjects,  in  order  to  maximize  ease  of  use. 

A  Sun  4/470  platform  is  used,  running  SunOS  4.1.3.  The  workstation  has  64  MB 
of  system  memory.  Attached  to  the  workstation  is  a  parallel  transfer  disk,  capable  of 
storing  7-22  GB  of  digitized  mammograms,  and  restoring  these  graphic  files  to  be 
displayed  very  rapidly.  Indeed,  the  four  standard  views  of  an  initial  mammography 
examination  (MLO  +  CC  views  of  right  and  left  breast)  are  recalled  from  disk  and 
displayed  on  paired  monitors  in  less  than  5  seconds.  Using  an  average  image  size  of  7.5 
MB,  this  indicates  a  combined  transfer-display  rate  of  at  least  5  MB/sec.  The  two 
gray-scale  monitors  used  in  the  system  have  a  display  resolution  of  2048  pixels  by  2560 
scan  lines  in  portrait  mode  on  their  21 -inch  (diagonal)  screens,  with  10-bit  resolution. 
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Image  processing  is  performed  by  dedicated  hardware  (Pixar,  Inc),  enabling  an 
extensive  library  of  image  processing  routines  to  be  called  upon  for  region  of  interest 
(ROI)  investigations.  Specifically,  these  include  filtered  edge  detection,  contrast 
enhancement,  highlighted  calcifications,  and  relief  map  simulation.  In  addition,  images 
can  easily  be  manipulated  on  the  screen  using  real-time  window  and  level  controls  and 
ROI  magnification. 

The  only  major  limitation  we  have  encountered  so  far  in  our  digital  image  display 
system  is  a  reduced  ability  to  demonstrate  at  standard  resolution  (two  paired  images  per 
monitor)  the  most  subtle  mammographic  microcalcifications  seen  on  conventional  film 
mammograms.  We  encountered  this  limitation  when  digitizing  film  mammograms 
scanned  with  either  50-micron  or  100-micron  sampling  pitch  (using  the  specially 
designed  Abe  Sekkei  laser  film  scanner  described  in  our  grant  proposal).  This  has  caused 
us  to  exclude  cases  involving  extremely  subtle  microcalcifications  as  the  sole 
mammographic  finding  on  initial  four- view  examination  (fewer  than  1%  of  the  cases 
included  in  our  CFTF;  fewer  than  0.1%  of  cases  encountered  in  clinical  practice).  During 
Year  2,  we  moved  the  DBITF  workstation  from  its  previous  location  in  the  UCSF 
Informatics  Laboratory  to  the  UCSF  Mammography  Reading  Room,  where  residents, 
fellows,  visiting  radiologists,  and  UCSF  staff  radiologists  do  their  daily  work.  These 
various  physicians  have  been  using  the  DBITF  workstation  ever  since,  on  a  regular  basis, 
without  any  difficulty. 

Task  4.  Database  design  and  implementation.  During  Year  1,  we  completed 
preliminary  work  on  designing  the  architecture  of  the  teaching  file  database  system  and 
the  computer-aided  instruction  model,  exactly  as  outlined  in  our  grant  proposal.  In  this 
format,  we  assembled  a  series  of  20  DBITF  cases,  including  125  images  and  155 
interactive  query-and-instruction  questions.  The  bulk  of  this  work,  aided  by  feedback 
from  initial  test  subjects,  involved  the  development  of  a  fast,  robust,  reliable,  and 
user-friendly  interface.  The  20  cases  also  formed  the  basis  for  the  pilot  study  of  DBITF 
effectiveness  described  below. 

During  Year  2  we  completed  the  remaining  elements  of  database  design  and 
execution  that  had  not  been  accomplished  by  the  end  of  Year  1 .  Specifically,  we 
successfully  implemented  our  proposed  design  requiring  the  user  to  place  a  cursor  over 
each  lesion  to  be  identified  (pointing  and  clicking  with  a  trackball),  an  important  aspect 
of  response-driven  instruction,  since  only  in  this  manner  can  the  DBITF  reliably  test  the 
user’s  ability  to  perceive  abnormalities. 

Task  5.  Evaluation.  Although  our  initial  grant  proposal  called  for  this  activity  to  begin 
in  Year  3,  for  reasons  stated  previously  we  completed  a  pilot  test  of  DBITF  effectiveness 
and  user  acceptance. 

This  pilot  study  involved  1 8  of  the  20  cases  fully  integrated  into  the  DBITF,  with 
test  subjects  using  both  DBITF  and  CFTF  materials  [15].  We  studied  24  test  subjects 
(radiology  residents,  fellows,  and  general  diagnostic  radiologists  from  UCSF  and  a 
neighbor  institution),  who  were  randomly  assigned  to  start  with  either  the  DBITF  or  the 
CFTF.  Each  subject  was  given  a  pre-test  and  post-test,  involving  questions  about  breast 
imaging  and  breast  cancer  pertinent  to  the  case  material.  Subjects  then  completed  the 
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other  teaching  file,  to  serve  as  a  comparison.  The  time  required  to  complete  each 
teaching  file  was  recorded.  A  questionnaire  was  then  administered  asking  subjects  to  rate 
on  a  scale  of  1  to  10  the  ease  of  use,  level  of  enjoyment,  and  value  as  a  learning  tool  for 
both  the  DBITF  and  CFTF.  Subjects  also  were  asked  to  indicate  whether  they  would  be 
more  willing  to  use  the  DBITF  or  CFTF  in  the  future,  and  which  teaching  file  they  would 
recommend  to  a  colleague. 

The  mean  time  to  complete  the  18  cases  in  each  teaching  file  was  1.5  hours.  The 
group  taking  the  post-test  after  completing  the  DBITF  demonstrated  a  higher  mean 
improvement  (from  pre-test)  in  scores,  compared  to  the  group  that  initially  evaluated  the 
CFTF,  but  results  were  not  statistically  significant.  However,  significantly  higher  ratings 
(p  <  .01)  for  the  DBITF  were  recorded  regarding  ease  of  use,  enjoyment,  and  value  as  a 
learning  tool.  22  of  24  (92%)  subjects  favored  using  the  DBITF  over  the  CFTF  in  the 
future,  and  23  of  24  (96%)  subjects  favored  recommending  the  DBITF  to  a  colleague. 

Although  not  included  in  our  original  grant  proposal,  during  Year  3  a  fully 
functional  DBITF  workstation  was  displayed  as  a  scientific  exhibit  at  the  week-long 
annual  meeting  of  the  Radiological  Society  of  North  America,  a  meeting  attended  by 
more  than  50,000  registrants  [16].  There  was  not  sufficient  time  for  such  a  large  number 
of  individuals  to  formally  evaluate  the  DBITF  workstation,  but  many  hundreds  of 
radiologists  used  it  and  provided  us  with  suggestions  for  improvements.  These 
assessments  gave  us  additional  guidance  in  further  revising  the  numerous  interactive 
response-driven  test  questions  that  we  had  already  developed,  and  that  we  continued  to 
develop,  in  order  to  maximize  the  teaching  effectiveness  of  our  case  material. 

The  bulk  of  our  evaluation  took  place  in  Year  3  of  the  project,  as  initially  proposed. 
In  order  to  recruit  a  sufficient  number  of  volunteers  to  formally  evaluate  the  DBITF 
versus  the  CFTF,  we  had  to  limit  the  number  of  studied  cases  so  that  each  subject  could 
complete  his/her  evaluation  within  one  day.  We  selected  50-case  DBITF  and  CFTF 
modules  for  study,  on  the  basis  of  experience  gained  during  Year  2  indicating  that  the 
average  user  could  readily  complete  two  such  modules  during  one  day.  Each  50-case 
module  involved  cases  of  comparable  degrees  of  variety,  subtlety,  and  difficulty.  We 
recruited  40  volunteer  radiology  residents,  fellows,  and  general  diagnostic  radiologists  to 
serve  as  test  subjects  in  evaluating  the  DBITF  in  comparison  with  the  CFTF.  Subjects 
were  randomly  assigned  to  start  with  either  the  DBITF  or  the  CFTF.  Each  subject  was 
given  a  50-question  pre-test  and  post-test,  involving  questions  about  breast  imaging  and 
breast  cancer  pertinent  to  the  case  material.  Subjects  then  completed  the  other  teaching 
file,  to  serve  as  a  comparison.  The  time  required  to  complete  each  teaching  file  was 
recorded.  A  questionnaire  was  then  administered  asking  subjects  to  rate  on  a  scale  of  1  to 
10  the  ease  of  use,  level  of  enjoyment,  and  value  as  a  learning  tool  for  both  the  DBITF 
and  CFTF.  Subjects  also  were  asked  to  indicate  whether  they  would  be  more  willing  to 
use  the  DBITF  or  CFTF  in  the  future,  and  which  teaching  file  they  would  recommend  to 
a  colleague. 

The  mean  time  to  complete  the  50  cases  in  the  DBITF  and  CFTF  was  3.9  hours  and 
4.4  hours,  respectively.  The  group  taking  the  post-test  after  completing  the  DBITF 
demonstrated  a  significantly  higher  mean  improvement  (from  pre-test)  in  scores, 
compared  to  the  group  that  initially  evaluated  the  CFTF  (p  <  .01).  Much  more 
significantly  higher  subjective  ratings  for  the  DBITF  were  recorded,  regarding  ease  of 
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use,  enjoyment,  and  value  as  a  learning  tool  (p  <  .001).  49  of  50  (98%)  subjects  favored 
using  the  DBITF  over  the  CFTF  in  the  future,  and  all  50  subjects  favored  recommending 
the  DBITF  over  the  CFTF  to  a  colleague. 

These  results  suggest  that  the  DBITF  is  an  effective  approach  to  teach  breast 
imaging  interpretation,  and  that  it  is  more  readily  accepted  by  radiologists  than 
conventional  film-based  teaching  files. 

Task  6.  Dissemination  of  the  DBITF.  During  Year  2,  the  DBITF  was  made  available 
to  all  radiology  residents,  fellows,  staff  radiologists,  and  visiting  radiologists  as  an 
optional  means  to  acquire  continuing  medical  education  at  UCSF.  At  the  start  of  Year  3, 
use  of  the  DBITF  became  a  mandatory  part  of  the  breast  imaging  instruction  provided  at 
UCSF;  indeed,  during  this  year  it  was  used  at  UCSF  by  more  than  300  radiologists  from 
all  over  the  United  States.  Also  during  Year  3,  we  disseminated  our  research  findings  to 
the  general  community  of  practicing  radiologists  in  the  form  of  an  article  published  in  the 
electronic  journal  of  the  Radiological  Society  of  North  America  [17].  This  is  now 
available  on  the  World  Wide  Web  (http://ej.rsna.org/EJ_0_96/0037-97.fin/mammo.html). 
It  contains  a  full  description  of  DBITF  design  and  implementation,  as  well  as  on-line 
demonstration  of  a  DBITF  case.  We  also  published  an  article  describing  in  detail  the 
digital  workstation  used  in  the  DBITF  [18].  Finally,  we  will  present  our  research 
findings  later  this  year  in  Washington,  DC,  at  the  Department  of  Defense  Breast  Cancer 
Research  Program’s  Era  of  Hope  meeting,  not  only  as  a  (required)  poster  presentation  but 
also  as  an  (invited)  platform  presentation. 

The  Breast  Imaging  Section  at  UCSF  will  maintain  the  DBITF  after  completion  of 
the  funded  research  project.  Teaching  file  cases  will  be  updated  periodically  by  replacing 
older  with  newer  case  material  (images,  question  and  answer  sets).  When  the  availability 
of  sophisticated  digital  breast  imaging  workstations,  such  as  that  which  we  have  created 
at  UCSF,  becomes  more  widespread  (including  the  necessary  computer  hardware  and 
high-resolution  display  monitors),  we  plan  to  distribute  the  DBITF  throughout  the  United 
States. 

CONCLUSIONS 

This  project  has  been  completed  successfully,  according  to  the  description  provided  in 
our  Statement  of  Work.  We  have  indeed  developed  a  functional,  user-friendly, 
full-featured,  comprehensive,  interactive  DBITF.  This  is  now  available,  at  UCSF,  as  a 
national  resource  for  the  teaching  of  breast  imaging  interpretation.  We  will  make  it 
available  throughout  the  United  States  as  soon  as  the  availability  of  (required)  digital 
breast  imaging  workstations  becomes  more  widespread. 
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RECOMMENDATION 

Although  we  plan  to  disseminate  the  DBITF  throughout  the  United  States  when  the 
availability  of  high-resolution  digital  workstations  becomes  more  widespread,  this  may 
take  several  years.  In  the  meanwhile,  the  Department  of  Defense  Breast  Cancer  Research 
Program  should  consider  supporting  the  continuation  of  funding  for  maintenance  of  the 
hardware  used  in  this  project  (a  service  contract  for  repair  and  replacement  of  all 
appropriate  equipment  costs  approximately  $9000  per  year). 
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